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Abstract 

A new framework for analyzing Fejer convergent algorithms is presented. Using this frame- 
work we define a very general class of Fejer convergent algorithms and establish its convergence 
properties. We also introduce a new definition of approximations of resolvents which preserve 
some useful features of the exact resolvent, and use this concept to present an unifying view of 
the Forward-Backward splitting method, Tseng's Modified Forward-Backward splitting method 
and Korpelevich's method. We show that methods based on families of approximate resolvents 
fall within the aforementioned class of Fejer convergent methods. We prove that such approximate 
resolvents are the iteration maps of the Hybrid Proximal-Extragradient method. 
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1 Introduction 

In this work we introduce a new framework for analysing Fejer convergent algorithms in Hilbert 
spaces, by means of recursive inclusions and sequences of point-to-set maps. This framework defines 
a new class of Fejer convergent methods, which is general enough to encompass, for example, the 
classical Forward-Backward splitting method, Tseng's Modified Forward-Backward splitting method 
and Korpelevich's method. Using this framework, we prove for good that convergence with summable 
errors is a generic property of a large class of Fejer convergent algorithms. Therefore, we regard this 
convergence result (with summable errors) as a rather negative result, in the sense that it is too 
generic to convey useful information on Fejer convergent methods. For sure, this kind of convergence 
for particular Fejer convergent algorithms lacks particular value. 

Another original contribution of this work is the concept of approximate resolvents of maximal 
monotone operators. Approximate resolvents retain the relevant features of exact resolvents as 
iteration maps for finding zeros of maximal monotone operators, their elements are more easily 
computable and are indeed calculated in, for instance, the classical Forward-Backward splitting 
method, Tseng's Modified Forward-Backward splitting method and Korpelevich's method. We prove 
that any algorithm based on approximate resolvents fall within the above mentioned class of Fejer 
convergent methods, providing an unifying framework for establishing their convergence properties. 

We present a new transportation formula for cocoercive operators and use it for establishing that 
the Forward-Backward splitting method is a particular case of the Hybrid Proximal-Extragradient 
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method. The relationship between the Hybrid Proximal-Extragradient method and approximate 
resolvents is also discussed. 

This work is organized as follows. In Section [21 we introduce some basic definitions and results. 
In Section [3J we define a very general class of Fejer convergent by means of recursive inclusions and 
sequences of point-to-set maps satisfying two basic properties. In Section HI we define approximate 
resolvents, show that they are the iteration maps of the Hybrid Proximal-Extragradient method 
and prove that methods based on approximate resolvents fall within the aforementioned class of 
Fejer convergent methods. In Section [5j we show that the Forward-Backward method is based 
on approximate resolvents, which is to say that it is a particular case of the Hybrid Proximal- 
Extragradient method. In Section [61 we recall that Tseng's Modified Forward-Backward method is 
a particular case of the Hybrid Proximal-Extragradient method, which is to say that it is based on 
approximate resolvents. In Section [7J we recall that Korpelevich's method is a particular case of the 
Hybrid Proximal-Extragradient method, which is to say that it is based on approximate resolvents. 
In Section [8l we make some comments. 

2 Basic definitions and results 

In the first part of this section, we review the concept and properties of Quasi-Fejer convergence, 
which will be used in our analysis of a class of Fejer convergent methods. In the second part, we es- 
tablish the notation concerning point-to-set maps, which will be used for defining the aforementioned 
class. 

The last part of this section contains the material needed to define approximate resolvents and 
the Hybrid Proximal-Extragradient (HPE) method, to prove that methods based on approximate 
resolvents/HPE method belong to the aforementioned class, and that some well known decomposition 
methods are based on approximate resolvents/HPE method. 

As far as we know, this section contains just one original result, namely, Lemma 12.81 which is a 
transportation formula for cocoercive operators. 

Quasi-Fejer convergence 

The concept of Quasi-Fejer convergence was introduce by Ermol'ev [9] in the context of sequences of 
random variables (see also [10] and its translation [H]). We will use a deterministic version of this 
notion, considered first in metric spaces by Isuem, Svaiter and Teboulle |12 | 113 1 Definition 4.1], in 
Euclidean spaces [H Definition 1], in Hilbert spaces in [2], in reflexive Banach spaces in pQ. 

All this material is now standard knowledge and is included for the sake of completeness. We 
do not claim to give here any original contribution to this over-studied concept. We will use an 
arbitrary exponent p just to unify the results related in the references; its particular value seems of 
little importance as indicated by the next results. 

Definition 2.1. Let X be a metric space and < p < oo. A sequence (x n ) in X is p-Quasi-Fejer 
convergent to Q C X if, for each x* € f2, there exists a non-negative, summable sequence (p n ) such 
that 

d(x*,x n ) p < d(x*,x n -i) p + p n n = 1,2, ... 

Note that if p\ = pi = • • • = in the above definition, we retrieve the classical definition of Fejer 
convergence and the exponent p becomes immaterial. Ermol'lev considered the stochastic case with 
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p = 2 and the deterministic case was considered in [12\ [T3] with p = 1 and in [H [2] with p = 2 in 
Euclidean and Hilbert spaces respectively. The next proposition summarizes the main properties of 
Quasi-Fejer convergent sequences in metric spaces. 

Proposition 2.2. Let X be a metric space, p S (0, oo) and (x n ) be a sequence in X which is 
p- Quasi-Fejer convergent to 0, C X then, 

1. if Q, is non-empty, then (x n ) is bounded; 

2. for any x* £ £1 there exists \\m. n ^ 00 d{x* ^x n ) < oo; 

3. if the sequence (x n ) has a cluster point x* G Q, then it converges to such a point. 
Proof. Take x* € f2 and let (p n ) be as in Definition 12.11 Then for n <m 

m 

d(x*,x m ) p < d(x*,x n ) p + ^ Pi- 

i=n+l 

Hence 

oo 

lim sup d(x*,x m ) p < d(x*,x n ) p + pi < oo, 

i=n+Y 

which proves item 1. To prove item 2, note that (p n ) is summable and take the limin^^oo at the 
right hand side of the first inequality in the above equation. Item 3 follows trivially from item 2. □ 

Now we recall Opial's Lemma [15], which is useful for analyzing Quasi-Fejer convergence in 
Hilbert spaces: 

Lemma 2.3 (Opial). If, in a Hilbert space X, the sequence (x n ) is weakly convergent to x* , then 
for any i/i* 

lim inf \\x n — x\\ > lim inf \\x n — x*\\. 

The next result was proved in [16], for the case of a specific sequence generated by an inexact 
proximal point method with p = 1, but the proof presented there is quite general, and we provide it 
here for the sake of completeness. The idea of using Opial's Lemma seems to be due to H. Brezis. 
Latter on this result was explicitly proved for Quasi-Fejer convergent sequences in Hilbert and Banach 
spaces with p = 2 in [21 Proposition 1], [lj Lemma 2.8] respectively. 

Proposition 2.4. //, in a Hilbert space X, the sequence (x n ) is p- Quasi-Fejer convergent to £1 C X, 
then it has at most one weak cluster point in $7. 

Proof. If x* G £1 is a weak cluster point of then there exists a subsequence (x nk ) weakly 

convergent to x*. Therefore, using item 2 of Proposition 12.21 and Opial's Lemma, we conclude that 
for any x' € f2, x' ^ x* 

lim \\x n — x'\\ = lim inf \\x n , — x II > lim inf ||x n — = lim \\x n — x*\\ 

n->oo k— >oo k— s-oo n— >oo 

which trivially implies the desired result. □ 
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It is trivial that the specific value of p G (0, oo) is immaterial in the above proofs. It would be 
preposterous to claim that for each p one has a "specific kind" of Quasi-Fejer convergence. We hope 
to reinforce this point of view with the next remark. 

Remark 2.5. Let X be a metric space, p G (0, oo) and (x n ) be a sequence in X which is p-Quasi- 
Fejer convergent to <Z X then either, 

1. d(x n ,x*) — > for some x* G Q; 

2. (x n ) is q-Quasi-Fejer convergent to Q for any q G (0, oo). 

Prom now on, p-Quasi-Fejer convergence will be called simply Quasi-Fejer convergence, the ex- 
ponent p being 1 unless otherwise stated. 

Point-to-set operators 

Let X, Y be arbitrary sets. A point-to-set map F : X =4 Y is a function F : X — > p(Y), where p(Y) 
is the power set of Y, that is, the family of all subsets of Y. If F(x) is a singleton for all x, that is, 
a set with just one element, one says that F is point-to-point. Whenever necessary, we will identify 
a point-to-point map F : X =$Y with the unique function / : X — > Y such that F(x) = {f(x)} for 
all x G X, 

A point-to-set map F : X =4 Y is L-Lipschitz if X and Y are normed vector spaces and, 

^ F(x') C {y + u | y G F(x), u G Y, \\u\\ < L\\x -x'\\}, Vx,a/eX. (1) 

Note that if F is point-to-point and it is identified with a function, then in the above definition we 
retrieve the classical notion of a L-Lipschitz continuous function. 

Maximal monotone operators and the ^-enlargement 

The e-enlargement of a maximal monotone operators will be used to define approximate resolvents 
in Section HJ In this section we review the definition of the e-enlargement and discuss those of its 
properties which will be used in the analysis and applications of approximate resolvents. 

From now on, X is a real Hilbert space. Recall that a point-to-set operator T : X z4 X is 
monotone if 

(x — y, u — v) > Vx, i/GX, u G T(x),v G T(y), 

and it is maximal monotone if it is monotone and maximal in the family of monotone operators in 
X with respect to the partial order of the inclusion. 

Let T : X X be a maximal monotone operator. Recall that the e-enlargement [5] of T is 
defined 

T^(x) = {v\(x-y,v-u)> -e}, x£X,e>0. (2) 

Now we state some elementary properties of the e-enlargement which follow trivially from the above 
definition and the basic properties of maximal monotone operators. Their proofs can be found 
in 0EI mi- 
Proposition 2.6. Let T : X X be maximal monotone. Then 
1. T = T[°]; 
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2. ifO < E\ < e 2 , then T^(x) C T^(x) for any x G X; 

3. A (rW(z)) = (AT)[ A£ l(x) for any x e X , e > and X > 0; 

4- if Vk G T^ £k \xk) for k = 1, 2, ... , (x/t) converges weakly to x, (v^) converges strongly to v and 
(efc) converges to e, then v G T^(x); 

5. ifT = df, where f is a proper closed convex function in X, then d £ f(x) C T^(x) = (df)^(x) 
for any x G X , e > 0. 

The e-enlargements of two operators can be "added" as follows. This fact was proved in [5] in a 
finite dimensional setting, but its extension to Hilbert and Banach spaces are straightforward. 

Proposition 2.7. It T\,T<i : X X are maximal monotone and T\ + T 2 is also maximal monotone 
then, for any £\,£2 > and x G X 

T[ £l] (x) + 4 £2 \x) C (Ti +T 2 )[ £1+£2 1(x). 

Recall that a (maximal) monotone operator A : X — > X is a-cocoercive (for a > 0) if 

(x — y, Ax — Ay) > a\\Ax — Ay\\ 2 , Vx, y G X. 

There is an interesting "transportation formula" for cocoercive operators. This result was proved by 
R.D.C Monteiro and myself. 

Lemma 2.8. // A: X — > X is a-cocoercive, then for any x, z G X, 

II l|2 

A(z)eA^(x), with e = A — . 

4a 

Proof. Take y £ X. Then 

(x — y, Az — Ay) = (x — z, Az — Ay) + (z — y, Az — Ay) 

> (x - z,Az - Ay) + a\\Az - Ay\\ 2 

> -\\x - z\\\\Az - Ay\\ + a\\Az - Ay\\ 2 , 

where the first inequality follows form the cocoercivity of A and the second one from Cauchy-Schwarz 
inequality. To end the proof, note that 

— Ilx — z\\ \\Az — Ay II + a\\Az — Ay\\ 2 > inf at 2 — ||x — z\\t 
and compute the value of the left hand-side of this inequality. □ 

The usefulness of the cr-approximate resolvent (to be defined in Section follows from the next 
elementary result, essentially proved in |17t Lemma 2.3, Corollary 4.2]. 

Lemma 2.9. Suppose that T : X X is maximal monotone, x G X , A > and a > 0. If 

v € T (y), z = x — Xv, 

\\Xv + y — x\f + 2Xe < a z ||y — x|| z , 

then \Xv\ < (1 + o~)\\y — x\\, \\z — y\\ < o~\\y — x\\ and for any x* G T -1 (0) 



II* Il2\ii* n2 I ii ||2 

\\x — x\\ > ||x — z\\ + \\y — x\\ - 

> \\x* - z\\ 2 + (1 - a 2 )\\y - x 



Xv + y - x\\ 2 + 2e 
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Proof. Since e > we have ||At> + y — x\\ < cr 1 1 2/ — x\\. The two first inequalities of the lemma follows 
trivially from this inequality, triangle inequality and the definition of z. 

To prove the third inequality of the lemma, take x* G T -1 (0). Direct combination of the algebraic 
identities 

|| x* — x\\ 2 = || x* — z\\ 2 + 2 (x* — y, z — x) + 2 (y — z, z — x) + \\z — x\\ 2 

11* n2 in/ * \ 1 11 u2 11 1 1 2 

= \\x — z\\ + 2 {x — y, z — x) + \\y — x\\ — \\y — z\\ 

with the definition of z yields 

\\x* — x\\ 2 = \\x* — z\\ 2 + 2A (x* — y, —v) + \\y — x\\ 2 — \\\v + y — x\\ 2 . 

Using the inclusions G T(x*), v G (y) and the definition in (J2J, we conclude that (x* — y, — v) > 
—e. To end the proof, of the third inequality, combine this inequality with the above equations. 
The last inequality follows trivially from the third one and the assumptions of the lemma. □ 



3 A class of Fejer convergent methods 

Let X be a Hilbert space and Q C X. We are concerned with iterative methods for solving problem 

xea (3) 

These methods, in their exact or inexact form, generate sequences (x n ) by means of the recursive 
inclusions 

x n G F n {x n -i) or x n G F n (x n -i) + r n , n = 1, 2, . . . , 

respectively, where F\ : X 14 X, F2 : X z4 X, . . . are point-to-set maps, r%,r2... are errors and 
xq G X is a starting point. The basic elements here are the set Q and the family of point-to-set maps 
(F n ), which we will call the family of iteration-maps. 

We will consider two properties of a general family of point-to-set maps (F n : X z4 X) ng N with 
respect to 0, C X: 

PI: if x G F n {x) and x* G O then 

II * A 1 1 ^ 1 1 * II 

If — x|| < ||x — x\\; 
P2: if (zk)keN converges weakly to z, z^ G F nk (zi t ) for n\ < ni < ■ ■ ■ and for some 

lim \\x* — Zfc|| — 1 1 a;* — %|| = 0, 

fc— >oo 

then z G £1. 

Property PI ensures that points in the image of F n {x) are closer (or no more distant) to than x. 
Regarding property P2, note that (using property PI) we have 

1 1 a;* — Zk\\ — \\x* — Zk\\ > 0. 

The left hand-side of the above inequality measures the progress of % toward the solution x*, as 
compared to z^. Hence, property P2 ensures that if the progress becomes "negligible", then the 
weak limit point of (zj~) belongs to Q. 
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Theorem 3.1. Suppose that 0, C X is non-empty and (F n : X n| X) is a sequence of point-to- set 
maps which satisfies conditions PI, P2 with respect to Q.. 
If 

Xn G Fn( x n-l) + r n, ^ ||r n [| < OO 

then (x n ) is Quasi-Fejer convergent to £1, it converges weakly to some i G and for any w G 
there exists linin^oo ||x* — x n \\. 

Moreover, if r n = for all n, then (x n ) is Fejer convergent to Q. 

Proof. To simplify the proof, define 

ry ry ly 

•^n — 1 n • 

Take an arbitrary x* G U. Since x n G F n (x n _i), \\x* — x n \\ < \\x* — x n _i||, 

I I ry ry I I <^ I I fY* ^ /"y 1 I I I I I ly I I I I ry ^ ry I I I I I ly I I 

^nW — 11*^ ^n|| ~r H'n| _ ||*^ ^n— 1\\ ~r ||'n|| 

and (x n ) nG N is Quasi-Fejer convergent to Q. Therefore, by Proposition 12.21 this sequence is bounded 
and there exists lim n _>. 00 ||x* — x n \\ < oo. Using this fact, the above equation and the assumption of 
(r n ) being summable we conclude that 

lim II 37* — x n _i|| — \\x* — x n \\ = 0. 

Since (x n ) ng N is bounded it has a weak cluster point, say x and there exists a subsequence {x nk )keN 
which converges weakly to x. The above equation shows that, in particular 

lim ||x* - x nfe _i|| - ||x* - x nk \\ = 

Therefore, using P2, the two above equations and the inclusion x nfe G F nfe (x rafc _i), we conclude that 
x G fi. Hence, all weak cluster points of (x n ) belong to Q. To end the proof, use Proposition 12.41 □ 

Note that properties PI, P2 are "inherited" by specializations. We state formally this result and 
the proof, being quite trivial, is be omitted 

Proposition 3.2. If (F n : X =t X) is a sequence of point-to-set maps which satisfies conditions PI, 
P2 with respect to Q C X and (G n : X z4 X) is a sequence of point-to-set maps such that, for any 
x G X 

G n {x) C F n (x), n = l,2,..., 
then (G n : X =3 X) also satisfies conditions PI, P2 with respect to fL 

What about compositions? Suppose that (F n ) is a sequence satisfying PI, P2 with respect to 
SI C X, and that 

F n = G n o H n 

where G n : X ^ Y, H n : Y =J X and all G n 's are L-Lipschitz continuous (Y is Hilbert). One may 
consider sequences 

Vn G H n (x n -\) + u n , x n G G n (y n ) + r n , (4) 
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where (u n ) and (r n ) are summable. Since x n — r n G G n (y n ), using also ([I]), we conclude that there 
exists x n e G n (y n - u n ) cG n o H n {x n -\) 

\\x n - (x n - r n )\\ < L \\u n \\. 

Therefore, defining s n = x n — x n we conclude that 

X n G F n (x n -i) + S n , ^ W S n\\ < X^'W + H rn H < °°- 

Therefore, if ^ 0, a sequence (x n ) generated as in (JH) converges weakly to some point x* G Q. 
On may also consider compositions of m + 1 maps 

= Gl, n ° G2, n • • • ° G m ^ n o H n 

adding summable errors in each stage, assuming each Gi tU : Y{ =4 to be L-Lipschitz continuous, 
H n :X^Y m ,Y = X etc. 



4 Approximate resolvents and the Hybrid Proximal- Extragradient 
Method 

In this section, first we define <r-approximate resolvents, analyze some of their properties and study 
conditions under which sequences of u-approximate resolvents satisfy properties PI, P2. After that, 
we recall the definition of the Hybrid Proximal-Extragradient method and show that cr-approximate 
resolvents are the iteration maps of such method. At the end of the section we discuss the incor- 
poration of summable errors to sequences of u-approximate resolvents and to the Hybrid Proximal- 
Extragradient method. 

Recall that the resolvent of a maximal monotone operator T : X =4 X is defined as 



J T (x) = {I + T)-\x), x£X. 
We shall consider approximations of the resolvent in the following sense. 

Definition 4.1. The cr-approximate resolvent of a maximal monotone operator T : X 
point-to-set operator Jt a X ^ X 



(5) 



X is the 



JtAx) 



3e > 0, y € X, 
v G T^(y) 



\v + y 



x\ 



+ 2s < a z \\y 



x\ 



where a > 0. 



First, we analyze some elementary properties of approximate resolvents and find a convenient 
expression for J\t,<j- In particular, we show that the cr-approximate resolvent is indeed and extension 
(in the sense of point-to-set maps) of the classical resolvent. 



Proposition 4.2. Let T : X 

1. Jt,it=o(x) = {Jt(x)}; 



X be maximal monotone. Then, for any x G X , 
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2. if < o\ < (J2 then Jt,o x (x) C Jt,<t 2 ( x ) > 

3. for any A > and a > 0, 



J\T,a(x) = { X - Xv 



3s>0,yeX, 
v G T^{y) 

\\Xv + y — x\\ 2 + 2Xe < a 2 \\y - x\\ 2 



Proof. Items 1, 2 and 3 follow trivially from Definition 14. II and Proposition 12.61 items [TJ [2] and [3l □ 

Note that in view of item 1 of the above proposition, if point-to-set operators which are point- 
to-point are identified with functions, we have 



In view of item 3, 



J. 



XT, a 



z€X; 



Jt,o = Jt ■ 



3£>0,2/Gl, 

^erH(y) 



\y — z\\ + 2Xe < cr \\y — x\ 



The next theorem is the main result of this section and states that, in some sense, approximate 
resolvents are "almost as good" as resolvents for finding zeros of maximal monotone operators, that 
is, for solving problem ([3]) with Q, = {x \ € T(x)} = T _1 (0). 

Theorem 4.3. Suppose that T : X X is maximal monotone, a € [0, 1), A > and (A&) is a 
sequence in [A, oo). Then, the sequence of point-to-set maps 

(J\ k T,a) 

satisfies properties PI, P2 with respect to Q, = {x G X \ S T(x)} = T -1 (0). 

Proof. Suppose that & G J\ k T,a( x )- This means that there exists y,v G X, e > such that 

x = x-X k v, DeT [£, (i), \\X k v + y - x\\ 2 + 2Ae < a 2 \\y - x\\ 2 . 
Therefore, using Lemma 12.91 we conclude that for any x* G= T _1 (0), 

II * ||2 \ ii * ^||2 I /-i 2\n ||2 \ ii * ^||2 

|F — x\\ > \\x — x\\ + (1 — a )\\y — x\\ > \\x — x\\ 

which proves that the family (J\ k T,a) satisfies PI. 

Now we prove P2. Suppose that (z k ) converges weakly to z, z k G J\ n T,a( z k), G T(x*) and 



lim — Z}~\ 

k— >oo 



\X - Z k \ 



0. 



To simplify the proof, let fi k = X nk > A. For each k there exists v k , yk G X, e k > such that 

z k = z k -fj, k v, v k G T £k (z k ), \\fi k v k + y k - z k \\ 2 + 2fi k e < a 2 \\y k - z k \\ 2 . 
Using again Lemma l2.9( we conclude that 

\\x* - z k \\ 2 > \\x* - z k \\ 2 + (1 - <r 2 )\\y k - z k \\ 2 . 



(6) 



(7) 
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Therefore 



/ -i 2mi ||2 ^ II * 1 1 2 || * - m2 

= (Ik* - z k\ - \\x* - z k \\){\\x* - z k \\ + \\x* - 4||). 

Since (z k ) is weakly convergent, it is also bounded. Taking this fact into account and using the above 
equation and ([6]) we conclude that 

lim \\y k - z k \\= 0. 

fc— >oo 

So, {y k ) also converges weakly to z. Since e k > 0, using the last relation in ([7]) we conclude that 

Hk^k < y ||yfc - z k \\ 2 , WfJ-kVkW < (1 + - ^fc||- 

Therefore, since (fi k ) is bounded away from 0, 

lim e k = 0, lim v k = 

andOeT[°](l) = T(l). □ 

The Hybrid Proximal- Extragradient /Projection methods were introduced in [18 } 1171 ^9]- These 
methods are variants of the Proximal Point method which use relative error tolerances for accepting 
inexact solutions of the proximal sub-problems. Here we are concerned with the variant introduced 
in [17], which will be called, from now on, the Hybrid Proximal- Extragradient (HPE) method. It 
solves iteratively the problem 

G T(x), (8) 
where T : X zz£ X is maximal monotone. This method proceeds as follows. 

Algorithm: (projection free) HPE method [TTj: 
Choose x G X, a G [0, 1), A > and for k = 1, 2, . . . 

a) Choose X k > X and find/compute v k , y k G X, e > such that 

■Ufc G T £k (y k ), \\X k v k + y k - x k -i\\ 2 + 2A fc £fc < a 2 \\y k - x fc _i|| 2 

b) Set x k = x k _ x - X k v k 

To generate iteratively sequences by means of approximate resolvents is equivalent to apply the 
HPE method in the following sense. 

Proposition 4.4. Let T : X X be maximal monotone, a > 0, X > and (X k ) be sequence in 
[A,oo). 

A sequence (x k ) satisfies the recurrent inclusion 

X k € J Xk T,a(x k -l), k = l,2, ... 

if and only if there exists sequences (y k ), (v k ), (e k ) which, together with the sequences (x k ), (X k ) 
satisfy steps a) and b) of the HPE method. 

Proof. Use Definition 14.11 and Proposition 14.21 item [51 □ 
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Convergence of the HPE method perturbed by a summable sequence of errors was proved directly 
in [BJ. Here we see that it can be easily and effortlessly derived as a particular case of a generic 
convergence result, combining Proposition 14,41 with Theorem 14.31 

Corollary 4.5. IfT : X X is maximal monotone, r _1 (Q) / 0, A > 0, a G [0, 1), for k = 1,2,... 

A fe > A 

f fc G T[ £fc] (x fc ), || Afc^fc + x k - x k _ x \\ 2 + 2A fc e fc < o-\\x k - x fc _i|| 2 
x fc = x fc _i - A fc f fc + r k 

and \\ r k\\ < °°> then (x k ) (and (x k )) converges weakly to a point x £ T _1 (0). 

Corollary 4.6. // T : X =1 X is maximal monotone, T _1 (0) ^ 0, A>A>0, a £ [0,1), for 
k = l,2,... 

A > A fc > A 

v k G T^- £k \x k ), \\\ k v k + x k - x fc _!|| 2 + 2X k e k < a\\x k - x fc _i|| 2 
x k = - A fc (i; fc + r k ) 
and ^ H^fcll < oo, then (x k ) (and (x k )) converges weakly to a point x £ T _1 (0). 

5 The Forward- Backward splitting method 

We will prove in this section that the iteration maps of the Forward-Backward splitting method are 
specializations or selections of cr-approximate resolvents and the sequence of iteration maps satisfies 
properties PI, P2. Equivalently, the Forward-Backward splitting method is a particular instance 
of the HPE method. Observe that, as a consequence, sequences generated by the inexact Forward- 
Backward splitting methods with summable errors still converge weakly to solutions of the inclusion 
problem, if any. This convergence result was previously obtained in [8] by a detailed analysis of the 
Forward-Backward splitting method. Here we see that it can be easily and effortlessly derived as a 
particular case of a generic convergence result. 

The Forward-Backward Splitting method solves the inclusion problem 

€ (A + B)x 

where 

fl) A : X — > X is a-cocoercive, a > 0; 
f2) B : X =4 X is maximal monotone. 

This method proceeds as follows: 

Forward-Backward Splitting method 

0) Initialization: Choose < A < A < 2a and xo € X; 

1) for k = 1,2,... _ 

a) choose A^ € [A, A] and define 

x k = (I + \ k BY 1 (x k - Afe^(x fc _i)) 

= Jx kB o(I-X k A) (x fc _i). (9) 
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Note that the generic iteration map of the Forward-Backward method is 

J\B o (/ - XA) (10) 

with A = Afc in the A;-th iteration. 

Lemma 5.1. If A, B satisfy fl and f2 then, for any A > and x G X , 

J\B XA)(x) G J\(A+B),a(x), 



with a = y / A/(2a). 

Proof. Take i£X and let z = J\b ° (I — XA)(x). This means that 

b:=\- 1 (x-XA(x)-z) €B(z). 

Define e = \\x — z|| 2 /(4a), v = A(x) + b. Using Lemma 12.81 we conclude that A{x) G A^(z). 
Therefore, combining this result with these two definitions, the above equation, Proposition 12.71 and 
Proposition 12.61 item [T| we conclude that 

v G {A^ + b)(z) C{A + B)W(z), \\Xv + z - x\\ 2 + 2Ae = a 2 \\z - xf 
z = x — Xv, 

which, together with Proposition 14.21 item 3, proves the lemma. □ 

Corollary 5.2. Let A,B be as in fl, f2 and X, A and (Xk), (xk) be as in the Forward Backward 
method. Define 

*=y/\/(2a). 

Then < a < 1 and for any x G X 

J\ kB °{I ~X k A){x) G Jx k{A +B),a{x), A: = 1,2,... (11) 

In particular 

Xk = J\ k B ° (I ~ X k A)(x k ) G J\ k (A+B),a(xk), k = 1,2,... (12) 
and the sequence of maps (J\ k B ° (I — X k A)) satisfies properties PI, P2 with respect to (^4 + i?) -1 (0). 

Proof. The bounds for a follow trivially from its definition and the choices for A, A in the Forward- 
Backward method. 

Define a k = y X k /{2a) for k = 1, 2, . . . Since X k G [A, A] , < a k < a for all k. Therefore, using 
also Lemma 15. II and Proposition 14.21 item [21 we conclude that for any i£X 

J\ k B ° (I ~ X k A)(x) G J\ k (A+B),a k (x) C Jx k (A+B),a(%), k = 1,2, . . . 

The equality in f)12|) follows trivially from the definition of the Forward-Backward method, while the 
inclusion follows from the above equation. To end the proof, note that < A < A^ for all k, and use 
Theorem 14,3} Proposition 13.21 and the above equation. □ 
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Proposition 5.3. Let (Xk), (x k ) be sequences generated by the Forward- Backward Splitting method. 
Define 




Then < a < 1 and for k = 1, 2, . . . 

v k G (5 + ^)[ £fc ](x fc ), HAfcTJfc + Xfc || 2 + 2A fc e A; < a||a; fc -x fc _i|| 2 
x k = - X k v k . 

In particular, the Forward-Backward splitting method above defined is a particular case of the HPE 
method with a G (0, 1). 

Proof. See the proofs of Lemma 15.11 and Corollary 15.21 □ 

6 Tseng's Modified Forward-Backward splitting method 

In |17| it was proved that Tseng's Modified Forward-Backward splitting method |21| is a particular 
case of the HPE method. Here we will cast this result in the framework of approximate resolvents, 
prove that the iteration maps of the Tseng's Modified Forward-Backward splitting method are spe- 
cializations or selections of cr-approximate resolvents and that the sequence of its iteration maps 
satisfies properties PI, P2. Observe that, as a consequence, sequences generated by inexact Tseng's 
Modified Forward-Backward splitting method with summable errors still converge weakly to solu- 
tions of the inclusion problem, if any. This result follows also from the fact that Tseng's method is a 
particular case of the HPE (as proved in [IT]) and that the HPE with summable errors converges (as 
proved in |6j). Convergence of Tseng's Modified Forward-Backward splitting method with summable 
errors was obtained in [3] by a detailed analysis of the Tseng's Modified Forward-Backward splitting 
method. Here we see that this result can be easily and effortlessly derived as a particular case of a 
generic convergence result. 

In this section we consider the inclusion problem 

G (A + B)x 

where 



tl) A : X — > X is monotone and L-Lipschitz continuous (L > 0); 
t2) B : X X is maximal monotone. 



The exact Tseng's Modified Forward-Backward Splitting method (without the auxiliary projection 
step) proceeds as follows: 

Tseng's Modified Forward-Backward method [21] 
Choose 0<A<A<1/L and x G X; 
for k = 1,2,... 

a) choose A& G [A, A] and compute 

y k = (I + \ k BY l (x k _i - A fc A(x fc _i)), x k = - X k (A(y k ) - A(x k -i)). 
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In order to cast this method in the formalism of Section [31 define for A > 

H x :X^XxX, H x (x) = (x, J XB (x - XA{x))) (13) 

G x : X x X -> X, G x (x,y) = y- X(A(y) - A(x))) (14) 

Note that the second component of the (generic) operator H\ is J\b o (J — XA) which is the generic 
iteration map of the Forward-Backward method in (|10p. Trivially, 

Xk = G\ k o H Xk (xk-i). (15) 

The next two result were essentially proved in [17J, in the context of the Hybrid Proximal-Extragradient 
Method. We will state and prove it in the context of cr-approximate resolvents. 

Lemma 6.1. If A,B satisfy assumptions tl, t2, then, for any X > and x G X 

G x o H x (x) G Ja+b, ct (x) 

for a = AL. 

Proof. Take ifX and let 

V = J\b{x - XA{x)), z = y + X{A(y) - A(x)). 
Note that z = G\ o H\{x). Using the definition of y we have 

a : = A" 1 ^ - XA{x) - y) € 

Therefore, 

u := a + A(y) G (A + ||A« + x - y\\ 2 = \\X(A(y) - A(x))\\ 2 

< (AL) 2 ||y-x|| 2 , 

where the inequality follows from assumption t2). To end the proof, note that z = x — Xv. □ 

Corollary 6.2. Let A,B be as in tl, t2 and < A < A < 2a and (A/-), (xk) be as in Tseng's 
Modified Forward- Backward method. Define 

a = XL. 

Then < a < 1 and for any idl 

G ^ k oH \ k ( x ) G J \ k (A+B),a(x), k = l,2, ... (16) 

In particular 

Xk = G Xk oH Xk (x k -l) G J\ k (A+B),a(xk-l), k = 1,2,... (17) 
and the sequence of maps (G Xk o H Xk ) satisfies properties PI, P2 with respect to (A + i?) -1 (0). 
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Proof. The bounds for a follow trivially from its definition and the choices for A and A in Tseng's 
Forward-Backward method. 

Define Ok = XkL for k = 1, 2, . . . Since A& G [A, A] we have < at < cr for all fc. Therefore, using 
also Lemma 16. II and Proposition 14.21 item 12"! we conclude that for any x G X 

J\ k B °(I ~ A fc A)(x) G JA ft (A+B),<T fc (») C ^A fc (A+B) >0 -(a;) 5 fe = 1,2,... 

The equality in (|17p follows trivially from the definition of the Tseng's Modified Forward-Backward 
method, while the inclusion follows from the above equation. To end the proof, note that < A < A& 
for all k, and use Theorem 14.31 Proposition 13.21 and the above equation. □ 

Note that for < A < A, the maps H\, G\ are Lipschitz continuous with constant 

2 + XL, 1 + 2AL, 

respectively. Hence, this method can be perturbed by summable sequences of errors in the evaluations 
of the resolvents J\ k B and/or in the evaluation of A(xk), A(yk) etc, and will still converge weakly to 
a solution, if any exists. 

7 Korpelevich's method 

In [14] it was proved that Korpelevich's method, with fixed stepsize, is a particular case of the HPE 
method. The extension of this result for variable stepsizes is trivial, and here we will analyze such 
an extension in the framework of approximate resolvents. Observe that, as a consequence, sequences 
generated by inexact Korpelevich's method with summable errors still converges weakly to solutions 
of the inclusion problem, if any. 

In this section we consider the inclusion problem 

G A(x) + N c {x) 

where 

kl) A : X — > X is monotone and L-Lipschitz continuous (L > 0); 

k2) Nq is the normal cone operator of C C X, a non-empty closed convex set. 

Korpelevich's method 

Choose 0<A<A<1/L and x G X; 

for k = 1,2,... 

a) choose A& G [A, A] and define 

Vk = Pc(xk-i ~ A fc F(aj fc _i)), x k = Pc(xk-i ~ hF(yk)), (18) 
where Pc stands for the orthogonal projection onto C. 

In order to cast this method in the formalism of Section [3l define for A > 

H x : X ->■ X x X, H x (x) = (x, P c {x - XA(x)), (19) 

G x :XxX^X, G x (x,y) = P c (x - XA(y)). (20) 
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Observe that since Pq = J\n c , the second component of the (generic) operator H\ is J\b o (I — XA) 
with B = Nq which is the generic iteration map of the forward backward method in (imp (with 
B = Nq)- Note also that the map H\ above defined has an equivalent expression 

H\(x) = (x,J XNc (x - XA(x))) 

which can be obtained by setting B = Nq in (|13p B = Nc- Trivially, 

x k = G Xk oH Xk (x k - 1 ), k = 1,2,... (21) 

The next two result were essentially proved in |14j . in the context of the Hybrid Proximal- Extragradient 
Method. We will state and prove them in the context of ^-approximate resolvents. 

Lemma 7.1. If A and C satisfy assumptions kl and k2, then, for any A > and x € X 

G\ o H x (x) £ Ja+N c ,(t (x) 

for a = XL. 

Proof. Take x £ X and let 

y = P c {x-XA(x)), z = P c {x-XA{y)). 
Note that z = G\o H\(x). Define 

rj = -(x - XA(x) - y), 

v = -{x - XA{y) - z), e=(u,z-y), v = u + A(y). 
Trivially, rj G Nc{y) and v € Nc{z) = d8c{z). Therefore, 

u£d £ 5c{y)d{d5c) [£] (y) = {Nc) [e] {y) 

and 

v G (A + N c ) [£] (y), z = x-Xv. (22) 

Therefore 

\\Xv + y - x\\ 2 + 2Xe = \\y - z\\ 2 + 2X(u, z - y) 

= h ~ z \? + 2 H V ~ V, z ~ y) + 2A(r?, z-y) 
< h ~ zf + 2X(v -n,z -y), 

where the inequality follows from the inclusions r\ € Nc(y), z G C. Direct algebraic manipulations 
yield 

\\y - z\\ 2 + 2X{u -n,z-y) = \\X(v - rj) + z - y\\ 2 - \\X(v - rf)\\ 2 

< \\X{v — rf) + z — y\\ 2 
= \\X{A{x) - A{y))\\ 2 . 

Combining the two above equations, and using assumption kl, we conclude that 

\\Xv + y- x\\ 2 + 2Xe < {XL) 2 \\y - x\\ 2 . 

The conclusion follows combining this inequality with (|22|) . □ 
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Corollary 7.2. Let A, C be as in kl, k2, and < A < A < 2a and (A&), (x&) be as in Korpelevich's 
method. Define 

a = XL. 

Then < a < 1 and for any x € X 

G X k oH \ k ( x ) G J x k (A+B),a(x), A; = 1,2,... 

In particular 

Xk = G\ k O H Xk (Xk-i) G J\ k (A+B),a (xk~i), k = 1, 2, . . . 
and £/ie sequence of maps (G\ k o iJ^ fc ) satisfies properties PI, P2 respect to (A + A r c) _1 (0). 

Proof. Use Lemma 17.11 and the same reasoning as in corollaries 15.21 and 16.21 □ 

Endowing X x X with the canonical inner product of Hilbert space products 

((x,y),(x',y')) = (x,x) + (y,y), 

it is trivial to check that for < A < A, the maps H\ and G\ are Lipschitz continuous with constants 

2 + AL, 1 + AL, 

respectively. Hence, one can analyze Korpelevich's method with (summable) errors in the projections 
and/or evaluations of A etc. 

8 Discussion 

We provided a general definition of generic methods by means of recursive inclusions and sequences of 
point-to-set maps. Using this formulation, we defined two properties of those maps which guarantee 
that the associated method is Fejer convergent and generates sequences which converge to a solution, 
if any, even when perturbed by summable errors. 

We think these results obviate the summable error convergence analysis of a number of convergent 
Fejer methods. 

The framework for the analysis of Fejer convergent methods introduced here is, of course, not 
general enough to encompasses all of these methods. Indeed, if X = M, Q = {0} and 




— x x > 0, 
x/2, x<0 



then any sequence (x n ) satisfying x n = F(x n -i) is Fejer convergent to {0} and converges to 0. 
However, the sequence (F n = F) does not satisfies P2. 

It has been since long recognized that Korpelevich's method (and may be even the Forward- 
Backward method) was an "inexact" version of the proximal point method. However, the nature 
and degree of this "inexactness" were not known. We provided a formal definition of approximate 
solutions of the prox by means of the <r-approximate resolvent which, while encompassing many 
classical decomposition schemes, also guarantees weak convergence of sequences generated by such 
approximate resolvents (even in the presence of additional summable errors). 
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